PVT: Point‐voxel transformer for point cloud learning

The recently developed pure transformer architectures have attained promising accuracy on point cloud learning benchmarks compared to convolutional neural networks. However, existing Transformers are computationally expensive because they waste a significant amount of time structuring irregular data. To solve this shortcoming, we present the Sparse Window Attention module gather coarse-grained local features from nonempty voxels. not only bypasses data and invalid empty voxel computation, but also obtains linear computational complexity with respect resolution. Meanwhile, leverage two different self-attention variants fine-grained about global shape according scale clouds. Finally, construct our architecture called point-voxel (PVT), which integrates these modules into joint framework for learning. Compared previous transformer-based attention-based models, method attains top 94.1% classification benchmark 10 × $10\times $ inference speedup average. Extensive experiments validate effectiveness PVT semantic segmentation benchmarks. Our code pretrained model avaliable at https://github.com/HaochengWan/PVT.